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ABSTRACT 



This paper describes Austin College's (Texas) participation 
in the Benchmark Project, a collaborative followup study of teacher education 
graduates and their principals, focusing on the second round of data 
collection. The Benchmark Project was a collaboration of 11 teacher 
preparation programs that gathered and analyzed data comparing graduates of 
4- and 5 -year programs and provided each program with data for program 
assessment. In 1996, Benchmark institutions surveyed their 1991-1995 
graduates and principals of those who were teaching. Data from the Austin 
Teacher Program (ATP) indicated that recent ATP graduates continued to enter 
the profession at higher rates than all Benchmark respondents. They expressed 
greater satisfaction with their career decision. They rated the value of the 
early field experiences far higher than did their counterparts. ATP graduates 
rated their teacher education program significantly higher than did the 
sample. They considered portfolio experience considerably less useful, and 
they rated their effectiveness lower than did their counterparts. The average 
rating for ATP students by principals on teaching effectiveness was very high 
(and consistently higher than for their counterparts) . Principals thought 
more highly of graduates' classroom management ability than the graduates 
themselves. (SM) 
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OBJECTIVE 

The objective of this paper is to describe Austin College's participation in 
the Benchmark Project, a collaborative follow up study of teacher education 
graduates and their principals, with particular attention to the second round of 
data collection. I will discuss the benefits of participation in Benchmark, 
including program assessment, institutional reaccreditation, recruiting and 
publicity, and enhancing the teacher education program's status on campus. I 
will also address the limitations of such follow-up studies, and the Austin 
Teacher Program’s tentative plans for future follow ups of its graduates. 
BACKGROUND 

Austin College, a private liberal arts college with an enrollment of 
approximately 1200, is located in Sherman, Texas, sixty-five miles north of 
Dallas. The college offers the Bachelor of Arts degree in a variety of arts and 
science disciplines, and the Master of Arts in Teaching. In 1971, Austin College 
implemented the five-year Austin Teacher Program (ATP), eliminating the four- 
year undergraduate certification program. Candidates for initial teacher 
certification must satisfactorily complete a sequence of three undergraduate 
education courses with field experiences (four semester hours each), and earn 
a B. A. before applying to the Graduate Program. The 36 semester hour M. A. T. 
includes professional education courses, content electives for secondary, and a 
teaching practicum, which may be graduate student teaching or a paid 
internship with full classroom responsibility. 
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THE BENCHMARK PROJECT 

Led by the University of New Hampshire in the 1990-1991 academic 
year, teacher preparation programs at eleven institutions from the northeast, 
south, and midwest began a collaborative follow up study of their graduates. 
(Initially called The Eleven Institution Project, members later adopted the name 
Benchmark Project.) The consortium's objective was to gather and analyze 
data that would not only compare graduates of four year and five year 
programs, but also would provide each program with useful data for program 
assessment. Membership in the Benchmark Project has changed over the 
years, but the number has staid at eleven. 

Representatives of several participating institutions designed and pilot 
tested a comprehensive survey of graduates' perceptions of their preparation, 
current instructional practices, beliefs about teaching, career plans, and 
professional activities. The investigators asked graduates' permission to send 
their principals a second instrument, later validated (Barton, Andrew & Schwab, 
1994), to rate their performance. Member institutions paid the University of New 
Hampshire to print and mail the surveys and to process the data. The first round 
of data collection on 1 986-90 graduates yielded results that indicated five year 
program graduates were significantly more likely to enter and remain in the 
profession, to participate in curriculum development, and to desire more time to 
collaborate with colleagues. While principals tended to rate all graduates high, 
ratings for teachers from five year preparation programs clustered at the top. 
(Baker, 1993) 

SECOND ROUND: THE ATP EXPERIENCE 

In 1996, Benchmark institutions surveyed their 1991-95 graduates and 
the principals of those who were teaching and gave permission. In this section, 
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I will describe how Austin College is using the those findings, citing examples 
rather than providing an exhaustive analysis. 

Participating institutions received computer printouts showing their own 
alumni's and their principals' responses, as well as aggregated data for the 
entire sample, permitting programs, if they wish, to compare their graduates' 
responses to a large sample of graduates from ten other respected programs. 

Graduates' Responses 

Recent ATP graduates continued to enter the profession at a higher rate 
than all Benchmark respondents ( 90.2% to 83.5%). They also expressed 
greater satisfaction with their career decision and greater willingness to enter 
teaching if they could start over ( 90% to 79.7%). Interestingly, then, ATP 
graduates' responses were virtually identical to the whole sample's when asked 
if they plan to be teaching in five years and ten years, but differences show up 
again when those answering no are asked about their future plans. Of all those 
saying they will not be teaching in five years, 31 .6% plan to be working outside 
the field of education compared to only 10% of ATP alumni. 

ATP graduates rated the value of their early field experiences far higher 
than the sample, with 59% calling them very helpful and 33.3% helpful. The 
corresponding percentages for the whole group were 6.4% and 39.9%. This 
may reflect the ATP's five year curriculum, which includes a sequence of 
undergraduate and graduate field experiences with increasing teaching 
responsibility before the graduate internship. When asked if their early field 
experiences were mostly observation, only 5.1% of ATP respondents agreed 
compared to 58.3% of the entire sample. An overwhelming 92.3% of ATP said 
their early field experiences consisted mostly of "active participation in teaching 
or teaching related activities", well ahead of the 38.2% for the whole group. 
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Responses to several other items indicated that ATP graduates feel 
particularly well prepared in general as well as in certain areas. Below are the 
percentages responding "very good" or "excellent". 





ATP 


ail 


Your effectiveness as change agent in school improvement. 


60.5 


51.5 


Your leadership in curriculum development 


55.3 


48.0 


Your leadership among peers 


65.9 


57.5 


How well did your program prepare you for 
leadership roles in schools? 


67.5 


26.0 


Overall rating of your teacher ed. program 


92.5 


51.5 



While the ATP faculty feels the graduates' responses generally affirm our 
program's structure and philosophy, it would be a mistake to think that all is 
perfect. We cannot learn from our former students unless we consider all their 
responses, the good and the not-as-good. Our graduates in this sample found 
significantly less value than their counterparts in creating a portfolio to 
demonstrate their pedagogical knowledge, a finding that led to discussions in 
our department. Faculty members discussed ways in which we could make the 
purpose and value of the portfolio clearer to students - and to our selves - , at 
the same time expanding it to become a continuing developmental portfolio 
from the second early field experience through the graduate internship. 

Another item that caused concern was our graduates' rating of their 
effectiveness in "teaching thinking skills necessary for independent learning." 
We assumed that our students would feel particularly strong in this area 
because they do a great deal of independent research in both the ATP and in 
their liberal arts program, yet only 47.3% said they were very good or excellent 
compared to 64.6% of the entire sample. Perhaps this reflects Texas' heavy 
emphasis on preparing elementary and secondary students for the Texas 
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Assessment of Academic Skills tests, making young teachers feel they have 
less latitude to use independent learning strategies. Or perhaps it signals a 
need for the ATP to address this area more strongly in its curriculum and its 
teaching strategies. Discussion continues. 

ATP grads were less enthusiastic about the helpfulness of their studies in 
historical and social foundations and philosophy of education than about their 
field experiences. While this in not surprising, - teacher education students 
tend to favor classroom experience over on-campus theory - it is cause for 
concern, even though graduates of the other institutions were not much more 
sanguine about those areas than ours. We have initiated discussions on ways 
to make our foundations course more authentic and more personal. 

Principal's Responses 

Eighty-six per cent of the ATP respondents who were teaching gave their 
permission to survey their principals. Again, we found the results encouraging. 
The average rating for ATP students on the 36 items related to teaching 
effectiveness was higher on most and about the same on the rest. On none 
were they lower. When asked, "Does this teacher stand our in some way from 
other teachers in your school," 100% of responding ATP principals said yes 
compared to 87% for the entire sample. Compared to teachers of similar 
experience, principals ranked 82% of the ATP grads in the top quintile, while 
only 62% of the entire group was so ranked. Following are selected items with 
the per cent of principals responding "very good" or "excellent". 





ATP 


Ml 


Competence in communication skills 


100 


88.5 


Planning & organizing lessons effectively 


100 


81 


Relating students' physical, social, emotional 
and intellectual development to planning instruction 


100 


85 
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Skill in classroom management 


90 


79.5 


Effective change agent in school improvement 


91 


74 


Leadership in curriculum development 


92 


70.5 


Leadership qualities among peers 


91 


70.7 



The item dealing with classroom management was especially interesting 
because ATP graduates' principals seem to think more highly of their ability in 
that area than do the graduates themselves. 

Skill in classroom management very good excellent 

ATP graduates' self-assessments 52.6 18.4 

ATP grads' principals’ ratings 30 60 

Perhaps beginning teachers feel they are supposed to be less skilled 
managers, or perhaps they are more aware minor annoyances that never come 
to the principal's attention. Yet the principals' ratings and the self-ratings' on 
management skills for the sample as a whole were almost identical. Without 
further investigation, one can only speculate on such a startling disparity on this 
item between ATP graduates' and their principals. The third ATP undergraduate 
course/field experience has included an intensive study of classroom 
management models since 1992, and discussions of management and 
discipline issues arise in our other classes and seminars. During numerous 
post-observation conferences, ATP students get personalized "coaching" on 
their management skills from ATP faculty members. Why only 18.4% believe 
they have excellent management skills is of some concern. We can take 
comfort in the principals' confidence in our graduates' management ability. 

BENEFITS OF PARTICIPATING IN THE BENCHMARK STUDY 
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Findings from the Benchmark Project have given us invaluable insight 
into the strengths and weaknesses of our program. It would be easy to merely 
rely on our students' scores on the ExCET (comprehensive tests of 
pedagogical and content knowledge required for licensure in Texas) to assess 
our program's effectiveness. ATP students always place at or near the top 
among the 66 teacher education programs in our state. But we would be 
missing the richest source of evaluative data, the perceptions of our graduates 
and of the principals who supervise them. We honor our clients -- our 
graduates and those who employ them -- by asking for their opinions of our 
program. Participation in the Benchmark Project also gives us opportunities to 
learn from our colleagues about innovations and promising practices at their 
institutions, and to compare our graduates perceptions and performance to a 
large, nationwide sample of teacher education graduates. 

There are other practical benefits. Austin College is preparing for a 
reaccreditation visit from the Southern Association of Colleges and Schools 
(SACS) in 1999. For the last two years, we have heard a great deal about 
SACS' demand for proof that the institution assesses the outcomes of its 
educational programs, causing consternation and gnashing of teeth in most 
departments of this liberal arts college. The Education Department has been 
pleased to provide our administration with information on how we are using 
Benchmark data to assess and improve our program, enhancing our 
department's standing on campus. In fact, we have been pointed out as a 
model in a campus-wide faculty meeting. Our public relations staff has asked to 
use some of our findings to refute regional newspaper articles that claim many 
new teachers are poorly prepared, do not persist in the profession, and have 
little regard for their preparation programs. 





LIMITATIONS 



We recognize the limitations of follow-up studies such as this one. They 
rely heavily on self-reporting. Furthermore, the number of ATP graduates in the 
1991-95 pool was unusually small, and the number of responding principals 
smaller still. Our numbers have risen in the last couple of years to late 1970s 
levels, but our graduates will always be a tiny percentage of the total 
Benchmark population. 

There is also the danger of dislocation of the shoulder -- from patting 
ourselves on the back, first, for simply undertaking such a project. Second, it is 
tempting to look for what we want to see in the data, highlighting our strengths 
while explaining away or downplaying findings that baffle or trouble us. Having 
said that, I think we have nevertheless learned an enormous amount about our 
teacher education program from the Benchmark Project. 

THE ATP'S PLANS FOR THE FUTURE 

If the Benchmark Project can obtain funding from foundations or other 
sources, I believe it will continue to provide valuable findings, not only to its 
member institutions, but also to other teacher educators and policy makers. 
Whether Austin College will be able to continue in the project is uncertain, 
although our program's participation has had administrative support and 
encouragement from the highest levels. 

I make no claim that Benchmark participants were the first to study their 
graduates' perceptions of their preparation, though I am not aware of any follow 
up of principals using a validated instrument. Nor would I discourage programs 
from developing their own follow up procedures to meet their unique needs. 
National-Louis University, for example, recently surveyed their graduates from 
the last five years to examine the match between their perceptions of their 
teacher education program and proposed national standards. (Prokosch & 
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Olson, 1998). But whatever the future of Benchmark, I would urge all teacher 
education programs to look to it as a model for collaboration, for assessing 
program effectiveness, and for learning from both our clients and our 
colleagues. 
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